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Abstract 

The nonnegative rank of an entrywise nonnegative matrix A G M^^" is the smallest 
integer r such that A can be written as ^4 = UV where U £ M™^*" and V £ R'j_^" are 
both nonnegative. The nonnegative rank arises in different areas such as combinatorial 
optimization and communication complexity. Computing this quantity is NP-hard in 
general and it is thus important to find efficient bounding techniques especially in the 
context of the aforementioned applications. 

In this paper we propose a new lower bound on the nonnegative rank which, unlike 
existing lower bounds, does not explicitly rely on the matrix sparsity pattern and 
applies to nonnegative matrices with arbitrary support. Our lower bound is expressed 
as the solution of a copositive programming problem and can be relaxed to obtain 
polynomial-time computable lower bounds using semidefinite programming. The idea 
involves computing a certain nuclear norm with nonnegativity constraints which allows 
to lower bound the nonnegative rank, in the same way the standard nuclear norm 
gives lower bounds on the standard rank. We compare our lower bound with existing 
ones, and we show examples of matrices where our lower bound performs better than 
currently known ones. 



1 Introduction 

Given a nonnegative^ matrix A G R![^^", the nonnegative rank of A is the smallest integer 
r such that A can be factorized as A = UV where U E R^^'^ and V E are both 

nonnegative. The nonnegative rank of A is denoted by rank+(y4) and it is always contained 
between rank(y4) and min(n, m): 

rank(A) < rank_|_(yl) < min(n, m). 

The authors are with the Laboratory for hiformation and Decision Systems, Department of Electrical 
Engineering and Computer Science, Massachusetts hrstitute of Technology, Cambridge, MA 02f 39. Email: 
{hf awzi ,parrilo}@niit . edu. 

^Throughout the paper, a nonnegative matrix is a matrix whose entries are all nonnegative. 
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The nonnegative rank appears in different areas such as in combinatorial optimization 
[Yan91] and communication complexity [KN06, Lov90, LS09]. Indeed, in combinatorial op- 
timization a well-known result by Yannakakis [YanQl] shows that the nonnegative rank of a 
suitable matrix characterizes the smallest number of linear inequalities needed to represent 
a given polytope P C M'^. This quantity is very important in practice since it indicates 
whether linear programs over the polytope P can be solved efficiently or not (the complex- 
ity of algorithms such as interior-point methods explicitly depend on the number of linear 
inequalities in the linear program). Another application of the nonnegative rank is in com- 
munication complexity where one is interested in the minimum number of bits that need to 
be exchanged between two parties in order to compute a binary function f{x,y), assuming 
that initially each party holds only one of the two arguments of the function. This quantity 
is known as the communication complexity of / and is tightly related to the nonnegative 
rank of a suitable matrix constructed from the function / [LS09, LovQO]. Finally it was also 
recently observed [ZhalO, JSWZ12] that the logarithm of the nonnegative rank of a matrix 
A coincides exactly with the minimum number of bits that need to be exchanged between 
two parties in order to sample from the bivariate probability distribution p{x, y) = A^^y 
represented by the matrix A (assuming A is normalized so that Ylxy^x,y = !)• 

Unfortunately, the nonnegative rank is NP-hard to compute in general [Vav09], unlike 
the standard rank. Researchers have therefore developed techniques to find lower and upper 
bounds for this quantity. Existing lower bounds are not entirely satisfactory though, since 
in general they depend only on the sparsity pattern of the matrix A and not on the actual 
values of the entries; see e.g. [LS09, FKPTll]. As a consequence, the bounds may fail in 
general when all the entries of A are strictly positive. In fact most of the currently known 
lower bounds are actually lower bounds on the rectangle covering number (also known as the 
Boolean rank) which is a purely combinatorial notion of rank that is itself a lower bound on 
the nonnegative rank. The rectangle covering number of a nonnegative matrix A G M™^" is 
the smallest number of rectangles needed to cover the nonzero entries of A. More precisely 
it is the smallest integer r such that there exist rectangles Ri = li x Ji (1 [m] x [n] for 
i = 1, . . . ,r such that 

r 

support(y4) = [J-Ri, 

i=l 

where support(A) = {{i,j) G [m] x [n] : Aij ^ 0} is the set of indices of the nonzero 
entries of A. It is easy to see that the rectangle covering number is always smaller than or 
equal to the nonnegative rank of A (each nonnegative rank 1 term in a decomposition of A 
corresponds to a rectangle). Some of the well-known lower bounds on the nonnegative rank 
such as the fooling set method or the rectangle size method are in fact lower bounds on the 
rectangle covering number and they only depend on the sparsity pattern of A and not on 
the specific values of its entries. 

In this paper we present the first computable lower bound on the nonnegative rank that 
does not rely on the sparsity pattern and that is applicable to matrices that are strictly 
positive. Before we present our result, recall that a symmetric matrix M G M"^" is said to 
be copositive if Mx > for all x G . Our result can be summarized in the following: 



2 



Theorem 1. Let A G M™^" be a nonnegative matrix. Let i^+{A) be the optimal value of the 
following optimization program: 



max 



{A,W) : 



Then we have 



rank+{A) > 



I -W 



u^iA) 



copositive 



(2) 



where \\A\ 



F ■- 



V- ,Af- is the Frobenius norm of A. 



As presented, the lower bound of Equation (2) is difiicuh to compute since copositive 
programming is known to be NP-hard in general (see e.g., [DiirlO]). Fortunately however, 
copositive programs can be nicely approximated using semidefinite programming. For ex- 
ample one can obtain a lower bound to ^^{A) by solving the following semidefinite program: 



max 



{A,W) 



I -W 
-W^ I 



where denotes the cone of symmetric positive semidefinite matrices of size n + m, and 

j^^n+m dQYLotes the cone of symmetric nonnegative matrices of size n + m. Since in general the 
sum of a nonnegative matrix and a positive semidefinite matrix is copositive, we immediately 
see that jy+{A) > u^^^A) for any A, and thus this yields a polynomial-time computable lower 
bound to rank_|_(A): 



rank+(A) > 



\A\ 



One can in fact obtain tighter estimates of using semidefinite programming by con- 

sidering hierarchies of relaxations of the copositive cone, like e.g., the hierarchy developed 
in [ParOO]. This is discussed in more detail later in the paper. 

It is interesting to express the dual of the copositive program (1). The dual of the cone 
of copositive matrices is the cone of completely positive matrices. A symmetric matrix 
M is said to be completely positive if it admits a factorization M = BB^ where B is 
elementwise nonnegative. Note that a completely positive matrix is both nonnegative and 
positive semidefinite; however not every such matrix is necessarily completely positive (see 
e.g., [BSM03] for an example and for more information on completely positive matrices). 
The dual of the copositive program (1) defining v+i^A) is the following completely positive 
program (both programs give the same optimal value by strong duality): 



v^{A) 



mm 



-(trace(X) + trace(F)) 



X 
AT 



A 
Y 



completely positive 



We will revisit this completely positive program later in the paper in Section 2.2 when we 
discuss the relation between the quantity v+iA) and the nuclear norm. 
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Outline The paper is organized as follows: In Section 2 we give the proof of the lower 
bound of Theorem 1 and we also outline a connection between the quantity i^+iA) and the 
nuclear norm of a matrix. We then discuss computational issues and we see how to obtain 
semi definite programming relaxations of the quantity u+lA). In Section 3 we look at specific 
examples of matrices and we show that our lower bound can be greater than the currently 
known lower bounds (i.e., the plain rank lower bound and the rectangle covering number). 
In general however our lower bound is uncomparable to the existing lower bounds, i.e., it can 
be either greater or smaller. Among the examples we show that our lower bound is exact for 
the slack matrix of the hypercube, thus giving another proof that the extension complexity 
of the hypercube in n dimensions is equal to 2n (a combinatorial proof of this fact is given 
in [FKPTll]). 



Notations Throughout the paper 5" denotes the vector space of symmetric matrices of 
size n and 5" is the cone of positive semidefinite matrices of size n. The cone of nonnegative 
matrices of size mxn is denoted by J\f"^^"- (for square symmetric matrices of size n we simply 
write A^"). If X G iS" we write X ^ to say that X is positive semidefinite. Given two 
matrices X and Y, their inner product is defined as {X,Y) = trace(X"'"F) = J2i j -^i,j'^i,j■ 



2 The lower bound 

2.1 Proof of the lower bound 

In this section we prove the lower bound of Theorem 1 in a slightly more general form: 

Theorem 2. Let A e M^^" be a nonnegative matrix. Assume P E M™^™ and Q E M"^" 
are nonnegative symmetric matrices with strictly positive diagonal entries. Let z/+(y4;P, Q) 
be the optimal value of the following copositive program: 



v+{A-P,Q) 



max 



{A,W) 



P 



-w' 
Q 



copositive 



Then we have 



~ y^traceiATPAQ) J 



(3) 



(4) 



Note that Theorem 1 in the Introduction corresponds to the special case where P and Q 
are the identity matrices. The more general lower bound of Theorem 2 has the advantage of 
being invariant under diagonal scaling: namely ii A = D1AD2 is obtained from A by positive 
diagonal scaling (where Di and D2 are positive diagonal matrices), then the lower bound 
produced by the previous theorem for both quantities rank_|_(A) and rank_|_(74) (which are 
equal) will coincide, for adequate choices of P and Q. In fact it is easy to verify that if we 
choose P = D^^ and Q = then 



i^+iA;P,Q) 
'trace(lTpIg) 



WAWp 
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The dual of the copositive program defining z/+(A; P, Q) is the following completely positive 
program where the matrices P and Q enter as weighting matrices in the objective function: 



v^iA]P,Q)= min 



-(trace(PX) + trace(QF)) 



X A 
A'^ Y 



completely positive 



Proof of Theorem 2. We introduce the shorthand notations ||x||p = a/ Px when x G W]^ 
and WvWq = \/y^Qy when y G M" which are well defined by the assumptions on P and Q 
(note however that || • ||p and || • ||q are not necessarily norms in the usual sense). Let A be an 
m X n nonnegative matrix with nonnegative rank r and consider a factorization of A of the 
form A = UV where U G W^'''' and V G M^^". For 1 < z < r, let Ui = U.^i/WU.^p G be 
the i'th column of U normalized so that \\ui\\p = 1, and Vt = Vi^./\\Vi^.\\Q be the i'th row of V 
normalized so that \\vi\\Q = 1. Then we can write A = ^^^^ \uivj where ||Mj||p = WviWq = 1 
and Xi = ||f^-,j||p||V^t,-||Q > 0. We know by the Cauchy-Schwarz inequality that 

iAiiiy_ / ELiA. 



rank+(y4) = r > 



A 



We will now show separately that the numerator ||A||f is lower bounded by the quantity 
i^+{A; P, QY, and that the denominator ||A||2 is upper bounded by ti a.ce{ A'^ PAQ): 



Numerator: If W is such that 



p -w 
-W^ Q 



Ui 


T 


P -W 




Ui 






_-W^ Q 




yi_ 



and hence 



Thus we get 



uJWv, < l{\\ui\\l + 



is copositive then, since Ui,Vi > 0, we have: 
> 

= 1. 



i=l 



2=1 



Note that this is true for any W such that |^_^t is copositive and thus we obtain 

r 

u+{A;P,Q)<J2>^i- 

1=1 

Denominator: We now turn to find an upper bound on ^11=1 ^i- Observe that we have 

r 

trace(A^PAg) = {PA, AQ) = ^ {XiPuivJ , XjUjvjQ) 



Ui 



1=1 



> 



i=l 



2|| ||2 II ||2 



i=l 
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where in the last inequality we used the fact that uJPuj > and vjQvj > which is 
true since P and Q are nonnegative. 

Now if we combine the two points above we finally get the desired inequality 



rank_|_(y4) = r > 



|A||i 
IAII2 



> 



+ {A;P,Q) Y 
trace(ATpAQ) J 



□ 



2.2 Connection with nuclear norm 



In this section we discuss the connection between the quantity z^+(v4) and nuclear norms 
of linear operators [Jam87]. If A is an arbitrary (not necessarily nonnegative) matrix the 
nuclear norm of A is defined by: 



uiA) 



mm 



2\\Vi\\2 



A 



(5) 



It can be shown that the quantity i^iA) above is equal to the sum of the singular values 
o'i{A) + ■ ■ ■ + (Jr{A) of A, and in fact this is the most commonly encountered definition of 
nuclear norm. This latter characterization gives the following well-known lower bound on 
rank(y4) combining z/(y4) and ||v4||ir: 



rank(y4) > 



cTijA) + ■ ■ ■ + ar{A) 
/ai{Ay + --- + ^AAy 



HAIL 



The characterization of nuclear norm given in Equation (5) can be very easily adapted to 
nonnegative factorizations of A by restricting the vectors Ui and Vi in the decomposition of A 
to be nonnegative. The new quantity that we obtain with this restriction is in fact nothing 
but the quantity i^+iA) introduced earlier, as we show in the Theorem below. Observe that 
this quantity z^+(v4) is always greater than or equal than the standard nuclear norm z/(y4). 

Theorem 3. Let A G W^^'^ be a nonnegative matrix. The following three quantities are 
equal to i^+{A) as defined in Equation (1).- 



(1) min I llwilhll^^ilh : A = ^Uivl, Ui,Vi - ^ | 



X m 



(2) rnin <| -{trace{X) + trace{Y)) : 



X A 
A'^ Y 



completely positive 



(3) max <^ iA,W) : 



/ -W' 
-W'^ I 



copositive 
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Proof. The conic programs in (2) and (3) are dual of each other and equahty of their optimal 
values follows from strong duality (both programs are easily seen to be strictly feasible) . We 
thus have to show only that (1) and (2) are equal. 

We start by proving that (1) < (2): If A = J^'^i'^J^ then if we let 



X = J ^ ^ Uiul and Y = 



Ui 2 



then 



X A 
A'^ Y 



2m\\2 



Ui/\\Ui\\2 



\Ui\\2 ^ 
\Vi\\2 



Ui/\\Ui\\2 



and so [ ^ y ] is completely positive since Ui and Vi are elementwise nonnegative. Further- 
more we have 

trace(X) + trace(F) = ^ ||^^i||2||fi||2 + ^ ||Mj||2||fi||2 = 2 ^ ||ui||2||fi||2- 



Hence this shows that (1) < (2). 

We now show that (2) < (1). Assume that X and Y are such that y] is completely 
positive and consider a decomposition 



X A 
A'^ Y 



E 



H T 



where Xi and t/i are nonnegative. Then from this decomposition we have A = ^2- xiyj and 

1 



i \ i i / 

This shows that (2) < (1) and completes the proof. 



-(trace(X) + trace(F)). 



□ 



Observe that for an arbitrary (not necessarily nonnegative) matrix A the solution of the 
primal-dual pair of SDPs 



mmimize 

subject to 



i(trace(X) + trace(F)) 



X 
AT 



A 
Y 



maximize 
subject to 



(AW) 
I 



-w 
I 



positive semidefinite 



positive semidefinite 

' ' ' (6) 

can be obtained from the singular value decomposition of A. Namely ii A = UTjV'^ is 
a singular value decomposition of A, then X = UTXJ'^ , Y = VT.V'^ and W = UV'^ are 
optimal points of the SDPs above. An interesting question is to know whether the primal- 
dual pairs (2)-(3) of completely positive/copositive programs in Theorem 3 lead to a spe- 
cific nonnegative factorization of A with interesting properties like the SVD. In the next 
proposition we explore such properties and we give a relation between the optimal W of 
the copositive program and an optimal nonnegative decomposition A = UivJ satisfying 
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Proposition 1. Let A G R^^"" be a nonnegative matrix. Assume A = Yli^i'^i'^i' ^■^ o, 
nonnegative decomposition of A that satisfies 

i 

where Aj > and Ui G G have unit norm \\Ui\\2 = \\vi\\2 = 1- Let W be an optimal 

point in the copositive program (1). Then we have for all i, 

Ui = Il+{Wvi) and vi = n+(W^^-Uj) 

where n+(x) = max(x, 0) is the projection on the nonnegative orthant. 

The proposition above translates in tlie nonnegative world the fact that, for arbitrary 
matrices A, the optimal W in the semidefinite program (6) satisfies Ui = Wvi and Vi = W'^Ui 
were Ui and Vi are respectively the left and right singular vectors of A. 

Before proving the proposition we prove the following simple lemma: 

is copositive, then for any v G M" we 
have 

\\II+{WV)\\2 < \\V\\2 

where n+(x) = max(x,0) is the projection on the nonnegative orthant. 

Proof. Let v G M" and call u = Il^{Wv). By the copositivity assumption we have, since u 
and V are nonnegative: 

u 

V 

which can rewritten as: 

{u - Wv^iu - Wv) + v^{I - W'^W)v > 0. 
Since u = Il+{Wv) we get: 

||n+(iyt;) - Wv\\l + \\v\\l - \\Wv\\l > 0. 
Note that ||n+(Vrt;) — Vrt;||2 — ||H^f II2 = — ||n+(Vr'u)||2 and thus we get the desired inequality 

\\U+iWv)\\l<\\v\\l. 

□ 

Using this lemma we now prove Proposition 1: 

Proof of Proposition 1. Let A = ^i^iUivJ be a nonnegative decomposition of A that sat- 
isfies -^j = ^+{^) let W be an optimal point of the copositive program (1). Since 



Lemma 1. IfWE 



is such that 



I 



-W 
I 



I -w 

-W^ I 





u 




V 



> 
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/ -w' 

-W^ I 
since W is optimal we have 



is copositive and ||Mi||2 = \\vi\\2 = 1, we have for each i, uJWvi < 1. Now 



^ XiuJWv, = {A, W) = u+{A) = J2 

i i 

and hence for each i we have necessarily uJWvi = 1. Furthermore, we have the sequence of 
inequahties 

1 = uJWvi < uJU+{Wvi) < \\ui\\2\\U+{Wvi)\\2 < WuMvih = 1 (7) 

where in the first inequality we used that u > 0, then we used Cauchy-Schwarz inequality 
and for the third inequality we used Lemma 1. Since the left-hand side and the right-hand 
side of (7) are equal this shows that all the intermediate inequalities are in fact equalities. 
In particular by the equality case in Cauchy-Schwarz we have that Il+{Wvi) = piUi for some 
constant pi. But since ulli^{Wv,i) = 1 and ||Mj||2 = 1 we get that pi = 1. This shows finally 
that Ui = U+iWvi). To prove that Vi = Il+{W''^Ui) we use the same line of inequalities as in 
(7) starting from the fact that 1 = vJlV'^^Ui. □ 



2.3 Relaxations and semidefinite programming lower bounds 

In this section we describe how one can use semidefinite programming to obtain polynomial- 
time computable lower bounds on i>^{A), and thus on rank+(y4). We outline here the hierar- 
chy of relaxations of the copositive cone proposed by Parrilo in [ParOO] which is a particular 
instance of a sums-of-squares relaxation. 

Recall that a symmetric matrix M G iS" is copositive if x'^Mx > for all x > 0. 
An equivalent way of formulating this condition is to say that the following polynomial of 
degree 4 



2 2 



is globally nonnegative. The cone C of copositive matrices can thus be described as: 

C = < M E S^' : the polynomial Mj jX^x^ is nonnegative 
I i,j=i 



The k'th order relaxation of C proposed by Parrilo in [ParOO] is defined as: 



k 




Ct'^l = I M eS"" : ( a;- I f Mj^x.^x^ ) is a sums-of-squares 



) 



It is clear that for any k we have C'^^^ C C and also C''^' C C^'^"'"^', which shows that the 
sequence (C['^l)fceN forms a sequence of tighter and tighter inner approximations to the cone 
of copositive matrices. A crucial property of this hierarchy is that each cone C''^^ can be 
represented using linear matrix inequalities, which means that optimizing a linear function 
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over any of these cones is equivalent to a semidefinite program (the size of the semidefi- 
nite program will of course increase as k increases). An interesting feature concerning this 
hierarchy is that the relaxation of order k = corresponds to 

(-[0] =^n^ ^n^ 

i.e., Ct°l is the set of matrices that can be written as the sum of a nonnegative matrix 
and a positive semidefinite matrix (this particular relaxation was already mentioned in the 
introduction). 

Using these semidefinite programming relaxations, we can now compute lower bounds to 
rank+(A) using semidefinite programming: 

Theorem 4. Let A G M™^" be a nonnegative matrix. For /c G N, let u^^^A) be the optimal 
value of the following semidefinite programming problem: 



I 



W 
I 



pI\A) =max I {A,W) : 
where the cone C^^' is defined in Equation (8). Then we have 

rank+{A) > ^ + ^ ' 



(9) 



(10) 



where \\A\\p := \l^ijA^- is the Frobenius norm of A. 



As said earlier, for the level A; = we have C'^^ = A/"" + S"^ and so the quantity u^^{A) 
takes the more explicit form: 



uX\A) = max <^ {A,W) : 



/ -W 



G AT"- 



n+m 



As gets larger the quantity (z/_f^(A)/||y4||ir)^ will converge (from below) to (z/+(A)/||y4||ir)^ 
(where v+i^A) is defined by the copositive program (1)) and this quantity can be strictly 
smaller than rank+(A) as can be seen for instance in Example 6 given later in the paper. In 
general, the quantity (z/l|^^'(y4)/||74||p')^ may not converge to rank_|_(74), however it will always 
be a valid lower bound to rank+(A). In summary, we can write for any > 0: 

u{A) < uf{A) < i}'1\A) < u+{A) < v/rank+(A)||A||^ 

where v{A) is the standard nuclear norm of A. The lower bounds (10) are thus always 
greater than the lower bound that use the standard nuclear norm (in fact they can also be 
greater than the standard rank lower bound as we show in the examples later). 

The MATLAB script http://www.niit .edu/~hf awzi/lower_bound_nonnegative_rank.ni 
allows to compute the quantity v^'^{A) and the associated lower bound on rank_|_(A) for any 
nonnegative matrix A and relaxation order /c G N. The script uses the software YALMIP 
and its Sums-Of-Squares module in order to compute the quantity v^^{A) [Lof04, Lof09]. 
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Note that there is another relaxation of the copositive cone that has been proposed by 
de Klerk and Pasechnik in [KP02]. This relaxation is based on linear programming (instead 
of semidefinite programming) and it leads in general to smaller programs that can be solved 
more efficiently. However the convergence of the LP hierarchy is in general much slower than 
the SDP hierarchy and one typically needs to take large values of k (the hierarchy level) to 
obtain interesting bounds. 



Program size reduction when A is symmetric If the nonnegative matrix A is square 
and symmetric, the semidefinite programs that define iy^+\A) can be simplified to obtain 
smaller optimization problems that are easier to solve. For simplicity, we describe below in 
detail the case of = 0; the extensions to the general case can also be done. Recall that the 
(dual) program defining 2^+^ (A) is: 



minimize -(trace(X) + trace(y)) subject to 



X A 
A Y 



e 5^" n AT' 



2n 



where 5^" n A/'^" is the cone of symmetric matrices that are positive semidefinite and non- 
negative (also known as doubly nonnegative). When A is symmetric, one can always find an 
optimal solution of the SDP above where X = Y. Indeed if the matrix y] is feasible, then 
so is the matrix ^] which also achieves the same objective function and thus by averaging 
we get a feasible point with the same matrices on the diagonal and with the same objective 
function. Therefore for symmetric A, the quantity u^^^A) is given by the optimal value of: 



X A 
A X 



e ST n A^- 



minimize trace (X) subject to 
Now using the known fact that 

^0 ^ X-A^OandX + AhO 



■2n 



X A 
A X 



we can rewrite the program as: 

minimize trace (X) subject to X — A y 0, X + A y 0, X nonnegative 

This new program has two smaller positivity constraints, each of size n, and thus can be 
solved more efficiently than the previous definition which involved one large positivity con- 
straint of size 2n. 

One can also obtain a reduction for the primal program that defines u^^^A), and this is 
given by: 

maximize tracelAW) subject to R — W 0, R + W ^ 0, I — R nonnegative (12) 
Observe that the constraint in the program above implies that [_wt~Y] ^ .S^^+A^^n 

smce: 





-w' 




'I-R " 


-w 


I 




I-R 



+ 



R 

-W 



-W 
R 



where in the right-hand side the first matrix is nonnegative and the second one is positive 
semidefinite. 
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3 Examples 



In this section we apply our lower bound to some explicit matrices and we compare it to 
existing lower bounding techniques. 

Example 1. We start with an example where our lower bound exceeds the plain rank lower 
bound. Consider the following 4x4 nonnegative matrix from [CR93]: 



A 



110 
10 10 
10 1 
11 



The rank of this matrix is 3 and its nonnegative rank is 4 as was noted in [CR93]. We can 
compute the exact value of i/!['^(y4) for this matrix A and we get //^^'(A) = 4\/2 fti 5.65. We 
thus see that the lower bound is sharp for this matrix: 



4 = rank+(A) > 



\A) 



\A\ 



4. 



The optimal matrix W in the semidefinite program^ (11) for which z/||° (A) 
by: 

"1 1-1-1 
1-1 1-1 
-1 1-1 1 
-1-1 1 1 



{A, W) is given 



(13) 



Observe that the matrix W is obtained from A by replacing the ones with ^ and the 
zeros with The matrix W is feasible for the semidefinite program (11) and we have the 

into a nonnegative part and a positive semidefinite 



following decomposition of 
part: 









' 





1 








I 


-w 







1 

























1 















1 
































1 




I 













1 

























1 

















1 









nonnegative 



+ 



" 1 





-1 











1 -1 







-w 




-1 1 
















-1 





1 














1 





-1 











1 -1 












-1 1 











-1 





1 . 



positive semidefinite 



(14) 



^Note that the matrix considered in this example is symmetric, and so one could use the reduced semidefi- 
nite program (12) to simplify the computation of lyf^ (A). However for simplicity and for illustration purposes, 
we used in this example the original formulation (11). 
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Example 2 (Slack matrix of the liypercube). The 4x4 matrix of the previous example 
is in fact the slack matrix of the square [0, 1]^ in the plane. Recall that the slack matrix 
of a polytope P C M" with / facet inequalities 6, — ajx > 0, i = 1,..., / and v vertices 
Xi, . . . ,Xv e is the f x v matrix S{P) given by: 



SiP)i,j = bi- ajxj Vi = 1, . . . , /, j = 1 



The matrix S{P) is clearly nonnegative since the vertices Xj belong to P and satisfy the facet 
inequalities. Yannakakis showed in [Yan91] that the nonnegative rank of S{P) coincides with 
the smallest number of linear inequalities needed to represent the polytope^ P. The minimal 
number of linear inequalities needed to represent P is also known as the extension complexity 
of P] the theorem of Yannakakis therefore states that rank_|_(5'(P)) is equal to the extension 
complexity of P. 

The hypercube [0, 1]" in n dimensions has 2n facets and 2" vertices. It is known that 
the extension complexity of the hypercube is equal to 2n and this was proved recently in 
[FKPTll, Proposition 5.9] using a combinatorial argument (in fact it was shown that any 
polytope in that is combinatorially equivalent to the hypercube has extension complexity 
2n). Note that the trivial lower bound obtained from the rank of the slack matrix in this 
case is n + 1; in fact for any full- dimensional polytope P C M", the rank of the slack matrix 
of P is equal to n + 1 (see e.g., [GRT12, Lemma 3.1]) and so the trivial rank lower bound is 
in general not interesting in the context of slack matrices of polytopes. 

Below we use the lower bound on the nonnegative rank introduced in this paper to give 
another proof that the extension complexity of the hypercube in n dimensions is 2n. 

Proposition 2. Let Cn = [0, 1]*^ be the hypercube in n dimensions and let S{C„ 
be its slack matrix. Then 



e R2nx2^ 



Proof. The proof is given in appendix A. □ 

Example 3 (Matrix with strictly positive entries). Consider the following matrix with 
strictly positive entries (e > 0): 



A, 



1+e 1+e e e 

1+e e 1+e e 

e 1+e e 1+e 

e e 1+el+e 



which is obtained from the matrix of Example 1 by adding a constant e > to each entry. 
For e = 0.1, we can compute the lower bound of Equation (10) with relaxation order k = 



■^Note that the trivial representation of P uses / hnear inequahties, where / is the number of facets of 
P. However by introducing new variables (i.e., allowing projections) one can sometimes reduce dramatically 
the number of inequalities needed to represent P. For example the cross-polytope P — {x ^ M" : x < 
1 Vw G {—1, 1}"} has 2" facets but can be represented using only 2n + 1 linear inequalities after introducing 
n additional variables: P = {a; G M" : 3y € y > x, y > — x, l^y < 1}. 
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and we get that 



rank+(Ao.i) > 



\\A 



3.2 



0.1 |F 



which shows that rank+(Ao.i) = 4. Note that the only existing nontrivial lower bound for 
strictly positive matrices is the plain rank lower bound and for the matrix Aq.i we have 
rank(Ao.i) = 3. 

Example 4 (Comparison with the Boolean rank lower bound). We now show an example 
where the lower bound (z/|''(A)/||a4||p')^ is strictly greater than the rectangle covering lower 
bound (i.e., Boolean rank lower bound). Consider the 4x4 matrix 

^0 1 1 r 

1111 
110 
110 

Note that the rectangle covering number of A is 2 since support(>l) can be covered with the 
two rectangles {1,2} x {2,3,4} and {2,3,4} x {1,2}. If we compute the quantity ^^^{A) 
and the associated lower bound we get rank4_(A) > [(z/!|'''(A)/||A||ir)^] = 3 which is strictly 
greater than the rectangle covering number. In fact rank+(/l) is exactly equal to 3 since we 
have the factorization 

"1 1 



The following simple proposition concerning diagonal matrices will be needed for the next 
example: 

Proposition 3. If A & R"^" is a nonnegative diagonal matrix and P is a diagonal matrix 
with strictly positive elements on the diagonal, then 

n 

u+{A;P,P)^{A,P) = J2A,,Pi,i. 
In particular, for P — I we get i^+{A) — trace{A). 



i=l 



Proof. Wc first show that u+iA; P, P) < {A, P): Observe that if W is such that 



P 

-W 



-W 
P 



is copositive, then we must have Wi^i < Pi^i for alH G {1, . . . , n} (indeed, take Cj to be the i'th 
element of the canonical basis of M", then by copositivity we must have 2eJ Pei — 2eJWei > 
which gives Wi^i < Pi,i). Hence, since A is diagonal we have {A,W) < {A,P) for all 
feasible matrices W and thus iy+{A; P, P) < {A, P). Now if we take W — P, we easily see 

^ ^ is copositive, and thus i/+(A, P;P) > {A,P). Thus we conclude that 



-W P 
v+{A-P,P) = {A,P). 



□ 
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Example 5 (Matrix rescaling). The well-known lower bound on the rank 

a,{l) + ■ ■ ■ + ariA) 

rank(A) > ' 



/ai{Ay + --- + ariAy 



is sharp when all the singular values of A are equal, but it is loose when the matrix is not 
well-conditioned. The same phenomenon is expected to happen also for the lower bound 
of Theorem 1 on the nonnegative rank, even if it is not clear how to define the notion of 
condition number in the context of nonnegative matrices. If one considers the following 
diagonal matrix where /3 > 

'(3 ... 0" 



A 











■•. 

1 



then we have from Proposition 3, i^+iA) = trace(y4) 
bound we get: 



P + n — 1, and thus for the lower 



rank_|_(y4) > 



MA) 
IIAIIf 



l3 + n-l 



n 



1 



If P is large and grows with n, say for example P = n, the lower bound on the right-hand 
side is < 0(1) whereas rank+^A) = n. 

To fix this one can use the more general lower bound of Theorem 2 with weight matrices 
P and Q to obtain a better lower bound, and in fact a sharp one. Indeed, if we let 



Q 



e 
1 



0" 



then we have i^+{A; P, Q)" 



[n 



_0 ... 

1 + /3e)2 and tmce{A'^PAQ) 



n 



1 + (/3e)^. Hence for 



1/(3, the ratio 



•^+{A;P,Q) 



is equal to n which is equal to rank+(A). 



^ytra.ce{A^PAQ) 

Example 6 (Derangement matrix). We now consider another example of a matrix that 
is not well-conditioned and where the lower bound of Theorem 1 is loose. Consider the 
derangement matrix 

"0 1 ... r 

Dr.. 

': ■•. ■•. 1 

_1 ... 1 0. 

that has zeros on the diagonal and ones everywhere on the off-diagonal entries. Then we 
have rank+(Dn) = rank(D„) = n, but we will show that 

2 

< 4 



n\\F 
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for all n. Observe that the matrix Dn is not well-conditioned since it has one singular value 
equal to n — 1 while the other remaining singular values are all equal to 1. 

To show that the lower bound of Theorem 1 is always bounded above by 4, note that the 
quantity u^{Dn) in this case is given by: 



v+iDn) = max { Wjj 



I -W 
-W^ I 



copositive 



Observe that by symmetry, one can restrict the matrix W in the program above to have 
the form^: 

'a b ... 6' 



W 



b . . . b a 



bJn + (a - b)I„ 



where J„ is the n x n all ones matrix. For W of the form above we have for u and v arbitrary 
vectors in R", u^Wv = {a — b)u^v + b{l''u){l''v) and hence the condition that ^ 



is copositive means that 



1 



Vm,wgM+, {a-b)u^v + b{l^u){l^v) < -{u^u + v'^v) 
Hence the quantity Vj^{Dn) can now be written as: 

v+{Dn) = max | {n^ - n)b : Wu,v e M" (a - b)u^v + b{l'^ u) {l'^ v) < -{u^u + v'^v) \ 

a,b£R I 2 1 

Let us call 6„ the largest b in the problem above, so that z/+(Z}„) = (n^ — Assuming n 
is even let 



Ur. 



:i, . . . , 1, 0, . . . , 0) G and = (0, . . . , 0, 1, . . . , 1) G 



n/2 



n/2 



Then we have ulvn = and the optimal bn must satisfy 6„(l"''M„)(l"'"f„) < + vJ^Vn), 

which gives bn^ < |, i.e., 6„ < 2/n. Hence Vj^{Dn) < (n^ — n) ■ 2/n = 2{n — 1) and 

4(1 -!/.)< 4. 



— n 



The case when n is odd can be treated the same way and we also obtain the same upper 
bound of 4. 



''indeed, observe first that the optimal W can be taken to be symmetric. Then for any feasible symmetric 
W and any permutation a, the new matrix W'^ j — Wo-(i),cr(j") is also feasible and has the same objective 
function as W. Hence by averaging we can assume W to be constant on the diagonal and constant on the 
oflF-diagonal. 
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4 Conclusion 



In this paper we have presented new lower bounds on the nonnegative rank that can be 
computed using semidefinite programming. Unhke previously known bounds, our lower 
bounds do not directly depend on the sparsity pattern of the matrix and are applicable to 
matrices with strictly positive entries. 

An interesting first question is to know whether the techniques presented here can be 
strengthened to obtain sharper lower bounds. In particular the results given here rely on 
the ratio of the £i and the £2 norms, however it is known that the ratio of the £1 and the ^p 
norms for 1 < p <2 can yield better lower bounds. Another interesting question is to know 
if the quantity v+i^) introduced in this paper can be related to approximate nonnegative 
factorizations or approximate nonnegative rank. 

A further direction that we are currently investigating are extensions of the techniques 
introduced in this paper to obtain lower bounds on the positive semidefinite (psd) rank. 
The psd rank was recently introduced in [GPTll], and has important applications in opti- 
mization [GPTll, FMP'^12], quantum communication complexity [FMP'^12] and quantum 
information [JSWZ12]. 

A Proof of Proposition 2: Slack matrix of hypercube 

In this appendix we prove Proposition 2 concerning the nonnegative rank of the slack matrix 
of the hypercube. We restate the proposition below for convenience: 

Proposition. Let Cn = [0, 1]" he the hypercube in n dimensions and let S{Cn) G ]R2nx2" 
its slack matrix. Then 



Proof. The facets of the hypercube C„ = [0, l]" are given by the linear inequalities {xk > 
0}, k = 1, . . . , n and {xk < 1}, k = 1, . . . ,n, and the vertices of Cn are given by the set 
{0, 1}" of binary words of length n. It is easy to see that the slack matrix of the hypercube 
is a 0/1 matrix: in fact, for a given facet F and vertex V, the {F, V^)'th entry of S{Cn) is 
given by: 



To show that we indeed have equality, we exhibit a particular feasible point W of the semidef- 
inite program (11) and we show that this W satisfies {{S{Cn),W) /WS^CnjllF)"^ = 2n. 





Since the slack matrix of the hypercube S{Cn) has 2n rows we clearly have 
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Let 



W 



1 



V2^ 



-.{2S{Cn) - J). 



Observe that W is the matrix obtained from S{Cn) by changing the ones into :^/=t 
zeros into ~^j^Tf- For this W we verify using a simple calculation that 



(15) 
and 



( {S{Cn),W) 
V \\S{Cn)\\F 



2n. 



It thus remains to prove that W is indeed a feasible point of the semidefinite program (11) 
and that the matrix 



)(2n+2")x(2n+2") 



can be written as the sum of a nonnegative matrix and a positive semidefinite one. This is 
the main part of the proof and for this we introduce some notations. Let J-" be the set of 
facets of the hypercube, and V = {0, 1}" be the set of vertices. If F G J-" is a facet of the 
hypercube, we denote by F the opposite facet to F (namely, if F is given by > 0, then 
F is the facet < 1 and vice- versa). Similarly for a vertex G V we denote by V the 
opposite vertex obtained by negating the binary word V. Denote by Njr : 



Nv 



the "negation" maps, so that we have: 



^geR^, VFg^, {N^g){F) 
yheR^,W e V, {Nvh){V) 



h{V) 



and 

(16) 
(17) 



Note that in a suitable ordering of the facets and the vertices, the matrix representation of 
Njr and Ny take the following antidiagonal form {Njr is of size 2n x 2n and A^v is of size 
2" X 2"): 

To 11 



1 

Consider now the following decomposition of the matrix 



I -W 



I -W 





A^v 



+ 



I-N^ -W 
-W^ I-Nv 



;i8) 



Clearly the first matrix in the decomposition is nonnegative. The next lemma states that 
the second matrix is actually positive semidefinite: 

Lemma 2. Let T and V be respectively the set of facets and vertices of the hypercube Cn = 
[0, 1]" and let W G R^""^ be the matrix: 



W, 



Fy 



1 ifV ^F 
-1 ifVeF 



yF eT, V eV 
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Then the matrix 



-7l?^ J-iVv 



(19) 



is positive semidefinite for 7 = l/v2^^ (where Njr and N\; are defined in (16) and (17) ). 

Proof. We use the Schur complement to show that the matrix (19) is positive semidefinite. 
In fact we show that 

1. / - iVv ^ 

2. range(iy"'") C range(/ — A^v), and 

3. I-N^- -f^W{I - Nv)-^W'^ >z 0. 

where (J — A^v)~^ denotes the pseudo-inverse of / — A'y. 
Observe that for any G N, the 2k x 2k matrix given by: 



2k 



-1 



-1' 



1 



is positive semidefinite: in fact ^{hk — -^2fc) is the orthogonal projection onto the subspace 
spanned by its columns (i.e., the subspace of dimension k spanned by {cj — 62^-1 : i = 
1, . . . , n} where Cj is the i'th unit vector). Hence this shows that / — A^v is positive semidef- 



inite, and it also shows that (/ — A'v 



Now we show that range(W^"'') C range(/ — A^v)- For any F E the F'th column of W'^ 
satisfies {W'^)v,f = —iW'^)vF ^ ^ V, and thus range(iy"'") C span(ey — cy, : V G 

V) = range (/-iVv). 

It thus remains to show that 

I -N^- ^^W{I - Nv)'^W'^ h 

First note that since ^{I — Ny) is an orthogonal projection and that range(iy"'") C range(/ — 
A^v)) we have (/ — Ny)~^W''^ = ^W''^ . Thus we now have to show that 

7' T 

The main observation here is that the matrix WW'^ is actually equal to 2"(/ — Njr). For 
any F,G E J-", we have: 



aev I 



if F = G 
if F = G 
else 
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Clearly if F = G, then {WW^)f,g = 2". Also if F = G then {WW^)f,g = -2'' (since if 
F = G then a e F <^ a ^ G hence Wp^aWca = -1 for all a G V). In the case that F ^ G 
and F 7^ G, it is easy to verify by simple counting that Xlaev ^F^oWca = 0. 

Hence we have I - - -f^W{I - Nv)-^W'^ = I - - ^WW^ = (1 - ^2")(/ - Nj) 
which is positive semidefinite for 7 = 1/ V2"~^. 

□ 

Using this lemma, Equation (18) shows that the matrix W is feasible for the semidef- 
inite program (11), and thus that v'^iSiCrS) — {S{Gn),W) = y/2^~^ ■ 2n. Hence since 
\\S{Gn)\\F = V2"-i ■ 2n we get that 

rank (S(G )) > ( '^±^^^]\ ( - 2n 
rank+(5(G„)) > |^ ^^^^^^^^^^ j > j - 2n 

which completes the proof. □ 
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